Orientation adjustable multi-channel haptic device

ABSTRACT

A system that generates haptic effects on a haptically-enabled device determines an orientation of the haptically-enabled device and obtains one or more haptic effect channels. The system then assigns each of the haptic effect channels to a haptic output device on the haptically-enabled device based on the orientation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/030,181, filed on Sep. 18, 2013, the specification of which is herebyincorporated by reference.

FIELD

One embodiment is directed generally to haptic effects, and inparticular to haptic effects generated by a multi-channel device.

BACKGROUND INFORMATION

Portable/mobile electronic devices, such as mobile phones, smartphones,camera phones, cameras, personal digital assistants (“PDA”s), etc.,typically include output mechanisms to alert the user of certain eventsthat occur with respect to the devices. For example, a cell phonenormally includes a speaker for audibly notifying the user of anincoming telephone call event. The audible signal may include specificringtones, musical tunes, sound effects, etc. In addition, cell phonesmay include display screens that can be used to visually notify theusers of incoming phone calls.

In some mobile devices, kinesthetic feedback (such as active andresistive force feedback) and/or tactile feedback (such as vibration,texture, and heat) is also provided to the user, more generally knowncollectively as “haptic feedback” or “haptic effects”. Haptic feedbackcan provide cues that enhance and simplify the user interface.Specifically, vibration effects, or vibrotactile haptic effects, may beuseful in providing cues to users of electronic devices to alert theuser to specific events, or provide realistic feedback to create greatersensory immersion within a simulated or virtual environment.

SUMMARY

One embodiment is a system that generates haptic effects on ahaptically-enabled device. The system determines an orientation of thehaptically-enabled device and obtains one or more haptic effectchannels. The system then assigns each of the haptic effect channels toa haptic output device on the haptically-enabled device based on theorientation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a haptically-enabled system in accordancewith one embodiment of the present invention.

FIG. 2 is a flow diagram of the functionality of the orientated hapticeffects module and the system of FIG. 1 when generating orientationadjustable haptic effect signals for actuators in accordance with oneembodiment.

FIG. 3 is a block diagram of an embodiment of the present invention thatuses sound to haptic conversion to generate haptic effect channels.

DETAILED DESCRIPTION

One embodiment is a haptically-enabled device/system that includes morethan one haptic channel. For example, the device can include a lefthaptic channel that generates haptic effects predominately on the leftside of the device, and a right haptic channel that generates hapticeffects substantially independent of the left haptic channel andpredominately on the right side of the device. The haptic device may behandheld/mobile and may change orientation (e.g., turned 180 degrees)during usage. Therefore, embodiments determine the current orientationand route the haptic channels accordingly so that they match up with thecurrent orientation. In general, embodiments map haptic signals withrespect to an actuator spatial arrangement of the device.

FIG. 1 is a block diagram of a haptically-enabled system 10 inaccordance with one embodiment of the present invention. System 10includes a touch sensitive surface 11 or other type of user interfacemounted within a housing 15, and may include mechanical keys/buttons 13.Internal to system 10 is a haptic feedback system that generatesvibrations on system 10. In one embodiment, the vibrations are generatedon touch surface 11.

The haptic feedback system includes a processor or controller 12.Coupled to processor 12 is a memory 20 and a left actuator drive circuit16, which is coupled to a left actuator 18. Actuator 18 may be any typeof actuator that can generate and output a haptic effect including, forexample, an electric motor, an electro-magnetic actuator, a voice coil,a linear resonant actuator, a piezoelectric actuator, a shape memoryalloy, an electro-active polymer, a solenoid, an eccentric rotating massmotor (“ERM”) or a linear resonant actuator (“LRA”).

Processor 12 may be any type of general purpose processor, or could be aprocessor specifically designed to provide haptic effects, such as anapplication-specific integrated circuit (“ASIC”). Processor 12 may bethe same processor that operates the entire system 10, or may be aseparate processor. Processor 12 can decide what haptic effects are tobe played and the order in which the effects are played based onhigh-level parameters. In general, the high-level parameters that definea particular haptic effect include magnitude, frequency, and duration.Low-level parameters such as streaming motor commands could also be usedto determine a particular haptic effect.

Processor 12 outputs the control signals to left actuator drive circuit16, which includes electronic components and circuitry used to supplyleft actuator 18 with the required electrical current and voltage (i.e.,“motor signals”) to cause the desired haptic effects. In addition,system 10 include a right actuator drive circuit 26 and a right actuator28, that operate substantially the same as the corresponding left sidedevices. Left actuator 18 can be positioned within system 10 to generatea vibratory haptic effect 30 predominantly on the left side of system10, and right actuator 28 can be positioned within system 10 to generatea vibratory haptic effect 40 predominantly on the right side of system10. System 10 further includes a sensor 25 that detects the orientationof system 10, such as an accelerometer, tilt sensor, three-dimensionaldetection sensor, etc. Signals from sensor 25 can be used by processor12 to determine the location or overall spatial arrangement of all ofthe haptic output devices of system 10.

In addition to or in place of actuators 18, 28, system 10 may includeother types of haptic output devices (not shown) that may benon-mechanical or non-vibratory devices such as devices that useelectrostatic friction (“ESF”), ultrasonic surface friction (“USF”),devices that induce acoustic radiation pressure with an ultrasonichaptic transducer, devices that use a haptic substrate and a flexible ordeformable surface or shape changing devices and that may be attached toa user's body, devices that provide projected haptic output such as apuff of air using an air jet, etc.

In other embodiments, actuators 18, 28 and sensor 25 may be in remotecommunication to processor 12. In these embodiments, processor 12 mayreceive signals from sensor 25, determine a mapping of haptic effectsbased on the signals, and transmit the haptic effects to thecorresponding remote haptic output devices. For example, processor 12and system 10 may be a central controller that controls and provideshaptic effects to wearable haptic devices such as wrist bands,headbands, eyeglasses, rings, leg bands, arrays integrated intoclothing, etc., or any other type of device that a user may wear on abody or can be held by a user and that is haptically enabled. Thewearable devices include one or more haptic output devices that generatehaptic effects on the wearable devices and are remote from system 10.

Memory 20 can be any type of storage device or computer-readable medium,such as random access memory (“RAM”) or read-only memory (“ROM”). Memory20 stores instructions executed by processor 12. Among the instructions,memory 20 includes an orientated haptic effects module 22 which areinstructions that, when executed by processor 12, generate orientationadjustable drive signals sent to drive circuits 16, 26 to generatehaptic effects, as disclosed in more detail below. Memory 20 may also belocated internal to processor 12, or be any combination of internal andexternal memory.

Touch surface 11 recognizes touches, and may also recognize the positionand magnitude of touches on the surface. The data corresponding to thetouches is sent to processor 12, or another processor within system 10,and processor 12 interprets the touches and in response generates hapticeffect signals. Touch surface 11 may sense touches using any sensingtechnology, including capacitive sensing, resistive sensing, surfaceacoustic wave sensing, pressure sensing, optical sensing, etc. Touchsurface 11 may sense multi-touch contacts and may be capable ofdistinguishing multiple touches that occur at the same time. Touchsurface 11 may be a touchscreen that generates and displays images forthe user to interact with, such as keys, dials, etc., or may be atouchpad with minimal or no images.

System 10 may be a handheld device, such a cellular telephone, personaldigital assistant (“PDA”), smartphone, computer tablet, gaming console,wearable device, or may be any other type of device that includes ahaptic effect system that includes one or more actuators. The userinterface may be a touch sensitive surface, or can be any other type ofuser interface such as a mouse, touchpad, mini-joystick, scroll wheel,trackball, game pads or game controllers, etc. In embodiments with morethan one actuator, each actuator may have a different rotationalcapability in order to create a wide range of haptic effects on thedevice. Not all elements illustrated in FIG. 1 will be included in eachembodiment of system 10. In many embodiments, only a subset of theelements are needed.

In one embodiment, system 10 is a multi-channel haptic device, meaningprocessor 12 generates more than one haptic effect channel (i.e., ahaptic effect signal that generates a haptic effect), and each channelis output/sent to a separate actuator or other haptic output device. Inone embodiment, system 10 generates a haptic effect channel thatcorresponds to each channel of audio data, such as the left and rightchannels of stereo audio data. In one embodiment, the haptic effectchannels/signals can be automatically generated from the audio channels,as disclosed in, for example, U.S. patent application Ser. Nos.13/365,984 and 13/366,010, the disclosures of which are hereinincorporated by reference.

On devices such as system 10 with multiple actuators, it is sometimesnecessary to change the haptic signals sent to the actuators based onthe posture or orientation of the device. For example, on system 10,actuators 18, 28 may be stereo piezo actuators, one on the left and oneon the right. The tactile effects in games and videos executed on system10 will typically be designed with the intention that some effects arefelt on the left and others are felt on the right. However most devicessuch as system 10 allow the user to flip the device completely around,and the visual image will spin to adapt so that it is stillright-side-up. Embodiments, therefore, flip the tactile effects so thatthe correct effects intended for the left side still play on the leftside, and the ones intended for the right side play on the right.

FIG. 2 is a flow diagram of the functionality of orientated hapticeffects module 22 and system 10 of FIG. 1 when generating orientationadjustable haptic effect signals for actuators 18, 28 in accordance withone embodiment. In one embodiment, the functionality of the flow diagramof FIG. 2 is implemented by software stored in memory or other computerreadable or tangible medium, and executed by a processor. In otherembodiments, the functionality may be performed by hardware (e.g.,through the use of an application specific integrated circuit (“ASIC”),a programmable gate array (“PGA”), a field programmable gate array(“FPGA”), etc.), or any combination of hardware and software.

At 202, the device orientation is determined based on one or moresensors, such as sensor 25. In one embodiment, sensor 25 is anaccelerometer.

At 204, the haptic effect data/channels are obtained. The haptic effectdata can be obtained directly from an application that generates thedata, or can be generated based on sound-to-haptic conversion techniquesor other conversion techniques. The haptic effect data includes multiplehaptic effect channels, where each channel is configured to be directedto a different haptic output device, such as a left or right actuator.

At 206, each haptic effect channel at 204 is assigned/mapped to anindividual haptic output device.

At 208, each haptic effect channel is sent to the corresponding assignedhaptic output device. The haptic output devices can be local or remotefrom system 10.

At 210, each haptic output device generates haptic effects in responseto receiving the haptic effect channel.

FIG. 3 is a block diagram of an embodiment of the present invention thatuses sound to haptic conversion to generate haptic effect channels. Inthe embodiment of FIG. 3, audio data is played at 300 and includes aleft audio channel and a right audio channel. However, in the example ofFIG. 3, because the orientation of the playback device (e.g., system 10of FIG. 1) has changed, the actuator 304 that was originally on the leftside is now on the right side, and vice versa. Therefore, it isdetermined that the left audio channel should be swapped with the rightaudio channel.

In Option 1, the audio channels can be swapped at 301, before beingreceived by a sound to haptic conversion module 310, which converts eachaudio channel to a haptic effect channel, and before being received by amixer 320. If Option 1 is used, a flag will be set in swap flag 330 toindicate that the channels have been swapped. Therefore, sound to hapticconversion module 310 can proceed to generate haptic effect channelswithout concern for the orientation of the device.

In Option 2, sound to haptic conversion module 310 receives un-swappeddata, and determines from swap flag 330 that the channels need to beswapped. Module 310 can then include the swapping functionality as partof the sound to haptic conversion functionality before outputting hapticchannels to the mapped actuators 340.

Although embodiments described above consider two directions whendetermining mapping (i.e., left and right), other embodiments canconsider four directions (i.e., top, bottom, left, right), or any othernumber of directions, depending on the number and placement of thehaptic output devices. Further, rather than left and right, the hapticoutput devices could be on the front and back of a device, or in someother arrangement. The mapping of haptic channels may also be based onhand position, grasp strength or grasp style (i.e., the way the user isholding the device) in addition to or in place of the orientation of thedevice. For example, if the user is tightly grasping the device on theleft side, while barely grasping the device on the right side, one ofthe haptic channels may be mapped to the tightly grasped side, and theother haptic channel may be mapped to the lightly grasped side. Further,the volumes or “magnitudes” of the haptic channels can be adjustedaccording to the grasps. For example, the side being tightly graspedcould be left at regular magnitude, while the side that is lightlygrasped could be increased in magnitude.

In some embodiments disclosed above, the device is rotated 180 degreesso that, for example, the left side is swapped with the right side.However, in some instances, a device with two actuators, one on eachside, may be rotated 90 degrees or some other amount less than 180degrees. In one embodiment, one or more of the followingmapping/assigning of channels may occur:

The left and right haptic channels are mixed into a single “center”channel to be played on both actuators.

No mixing—instead the left haptic effect is played on the actuator thatwas most recently on the left side, and the right haptic effect isplayed on what was the right actuator. This will provide the mostconsistent experience.

Some other attributes are used to determine which actuator to play on(e.g., one may be larger than the other).

In another embodiment with two actuators, the haptic signal is comprisedof four-channels (e.g., left/right/front/back), and the device isrotated 90 degrees or some other amount less than 180 degrees. In thisembodiment, the two channels that correspond to the current orientation(left/right OR front/back) are selected and those channels are rendered.The two off-axis channels are either dropped, or are mixed and treatedas a center channel to be played on one or both of the other actuators.

In general, when embodiments, are assigning tracks of a multi-channelhaptic effect to individual actuators, the first consideration is to usethe effects that match the positions/axes of the actuators in thesystem, followed by an optional mapping of off-axis effects to one ormore actuators.

Further, the haptic mapping may be modified according to where the useris touching a touch screen in a control widget. For example, a devicemay include a graphic touchscreen slider along the side of amulti-actuator device. The actuator closest to the user could receive ahaptic command signal, while the other actuators receive no hapticsignals.

The mapping may also apply to flexible devices and be dependent, forexample, on whether the screen is rolled up or stretched out, and mayapply to multi-cell touch screens.

Embodiments using mapping further include wearable haptic devices. Forexample, in one embodiment the haptic output device is a ring withmultiple vibration elements. The mapping of the haptic channels can beadjusted according to the orientation of the ring. For example, which ofthe actuators that is currently at the top of the ring will changedepending on the current orientation of the ring. When sending an “up”haptic signal (i.e., a haptic signal that imparts “up” information), thecurrent actuator on the top of the ring will receive the up hapticeffect. In another example, the user may be wearing a haptically-enabledwatch on each arm. If the user swaps the watches, system 10 willdetermine which arm has which watch and map the haptic effectsaccordingly.

Further, the mapping in embodiments can occur “mid-stream” such asduring the playing of a movie. For example, a user may be watching amovie that has stereo spatialized tactile effects on a tablet. The userpauses the movie and puts down the tablet. When the user returns andpicks up the tablet, it is in the opposite orientation as before. Thetablet rotates the display image to accommodate this. Further,embodiments also rotate the specialized haptic image by changing themapping.

Although embodiments disclosed above include multiple actuators, in oneembodiment the device can include a single actuator. In this embodiment,the haptic effect is changed based on how the user is holding the devicerather than the orientation of the device. For example, if the actuatoris on the left side, and the user is holding the device on the leftside, the magnitude of the haptic effect may be relatively weak.However, if sensor 25 detects that the user is holding the device on theright side, the magnitude of the haptic effect may be relatively strongso that the user can feel that haptic effect at a distance from theplacement of the actuator.

As disclosed, embodiments map haptic channels to actuators based on theorientation of the device, among other factors. Therefore, spatialhaptic effects will always be generated by the appropriate haptic outputdevice, regardless of how a mobile device, for example, is being held bya user.

Several embodiments are specifically illustrated and/or describedherein. However, it will be appreciated that modifications andvariations of the disclosed embodiments are covered by the aboveteachings and within the purview of the appended claims withoutdeparting from the spirit and intended scope of the invention.

What is claimed is:
 1. A non-transitory computer-readable medium havinginstructions stored thereon that, when executed by a processor, causethe processor to generate haptic effects on a haptically-enabled device,the generate haptic effects comprising: obtaining a haptic signalcomprising a first haptic effect channel, a second haptic effectchannel, a third haptic effect channel and a fourth haptic effectchannel; assigning the first haptic effect channel to a first hapticoutput device on the haptically-enabled device and assigning the secondhaptic effect channel to a second haptic output device on thehaptically-enabled device when a first orientation is determined; andassigning the third haptic effect channel to the first haptic outputdevice on the haptically-enabled device and assigning the fourth hapticeffect channel to the second haptic output device on thehaptically-enabled device when a second orientation is determined. 2.The computer-readable medium of claim 1, the generate haptic effectsfurther comprising: mixing the third haptic effect channel and thefourth haptic effect channel when the first orientation is determined togenerate a mixed channel, and assigning the mixed channels to at leastone of the first haptic output device or the second haptic outputdevice.
 3. The computer-readable medium of claim 1, the generate hapticeffects further comprising: not assigning the third haptic effectchannel and the fourth haptic effect channel to any haptic outputdevices on the haptically-enabled device when the first orientation isdetermined.
 4. The computer-readable medium of claim 1, wherein thehaptically-enabled device is a mobile device and the first haptic effectchannel corresponds to a left side of the mobile device, the secondhaptic effect channel corresponds to a right side of the mobile device,the third haptic effect channel corresponds to a front side of themobile device and the fourth haptic effect channel corresponds to a backside of the mobile device.
 5. The computer-readable medium of claim 1,wherein the first haptic output device comprises a first actuator andthe second haptic output device comprises a second actuator.
 6. Thecomputer-readable medium of claim 1, wherein the assigning comprisesremotely transmitting the haptic effect channels to thehaptically-enabled device.
 7. The computer-readable medium of claim 6,wherein the haptically-enabled device comprises a wearable device. 8.The computer-readable medium of claim 1, wherein the haptically-enableddevice comprises a flexible device, and the first orientation isdetermined based on whether the flexible device is rolled up.
 9. Acomputer implemented method to generate haptic effects on ahaptically-enabled device, the method comprising: obtaining a hapticsignal comprising a first haptic effect channel, a second haptic effectchannel, a third haptic effect channel and a fourth haptic effectchannel; assigning the first haptic effect channel to a first hapticoutput device on the haptically-enabled device and assigning the secondhaptic effect channel to a second haptic output device on thehaptically-enabled device when a first orientation is determined; andassigning the third haptic effect channel to the first haptic outputdevice on the haptically-enabled device and assigning the fourth hapticeffect channel to the second haptic output device on thehaptically-enabled device when a second orientation is determined. 10.The method of claim 9, further comprising: mixing the third hapticeffect channel and the fourth haptic effect channel when the firstorientation is determined to generate a mixed channel, and assigning themixed channel to at least one of the first haptic output device or thesecond haptic output device.
 11. The method of claim 9, furthercomprising: not assigning the third haptic effect channel and the fourthhaptic effect channel to any haptic output devices on thehaptically-enabled device when the first orientation is determined. 12.The method of claim 9, wherein the haptically-enabled device is a mobiledevice and the first haptic effect channel corresponds to a left side ofthe mobile device, the second haptic effect channel corresponds to aright side of the mobile device, the third haptic effect channelcorresponds to a front side of the mobile device and the fourth hapticeffect channel corresponds to a back side of the mobile device.
 13. Themethod of claim 9, wherein the first haptic output device comprises afirst actuator and the second haptic output device comprises a secondactuator.
 14. The method of claim 9, wherein the assigning comprisesremotely transmitting the haptic effect channels to thehaptically-enabled device.
 15. The method of claim 14, wherein thehaptically-enabled device comprises a wearable device.
 16. The method ofclaim 9, wherein the haptically-enabled device comprises a flexibledevice, and the first orientation is determined based on whether theflexible device is rolled up.
 17. A haptic system comprising: aprocessor; a first haptic output device coupled to the processor; asecond haptic output device coupled to the processor; a third hapticoutput device coupled to the processor; a fourth haptic output devicecoupled to the processor; an orientation sensor coupled to theprocessor; and a haptic effect mapper module coupled to the processorthat maps a haptic effect channel to a haptic output device based on anorientation of the system, the mapping comprising: obtaining a hapticsignal comprising a first haptic effect channel, a second haptic effectchannel, a third haptic effect channel and a fourth haptic effectchannel; assigning the first haptic effect channel to the first hapticoutput device and assigning the second haptic effect channel to thesecond haptic output device when a first orientation is determined; andassigning the third haptic effect channel to the first haptic outputdevice and assigning the fourth haptic effect channel to the secondhaptic output device when a second orientation is determined.
 18. Thehaptic system of claim 17, further comprising a transmitter coupled tothe processor; wherein the assigning comprises remotely transmitting thehaptic effect channels to the haptic output devices using thetransmitter.
 19. The haptic system of claim 18, wherein the hapticeffect channels are remotely transmitted to a wearable device.
 20. Thehaptic system of claim 18, the mapping further comprising not assigningthe third haptic effect channel and the fourth haptic effect channel toany haptic output devices on the haptic system when the firstorientation is determined.